Method and apparatus continuing action of user gestures performed upon a touch sensitive interactive display in simulation of inertia

ABSTRACT

A method and apparatus for identifying user gestures to control an interactive display identifies gestures based on a bounding box enclosing points at which a user contacts a touch sensor corresponding with the display surface and permits use of inexpensive and highly reliable grid-based touch sensors that provide a bounding box to describe contact information. In identifying gestures, position, motion, shape, and deformation of the bounding box may be considered. Center, width, height, aspect ratio, length and orientation of the bounding box diagonal may be determined. A stretch factor, defined as the maximum of the ratio of the height of the bounding box to the width of the bounding box and the ratio of the width of the bounding box to the height of the bounding box, may also be computed. Gestures may be identified based on the changes in time of these characteristics and quantities.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of the following earlier-filed andco-pending application and claims the benefit thereof in accordance with35 USC 120: U.S. patent application Ser. No. 13/458,915, filed Apr. 27,2012, which is a continuation of U.S. patent application Ser. No.12/862,564 entitled METHOD AND APPARATUS CONTINUING ACTION OF USERGESTURES PERFORMED UPON A TOUCH SENSITIVE INTERACTIVE DISPLAY INSIMULATION OF INERTIA, which was filed on Aug. 24, 2010 in the names ofW. Daniel Hillis and Bran Ferren, and issued on May 29, 2012 as U.S.Pat. No. 8,188,985. The '564 application is a continuation of U.S.patent application Ser. No. 11/188,186 entitled METHOD AND APPARATUSCONTINUING ACTION OF USER GESTURES PERFORMED UPON A TOUCH SENSITIVEINTERACTIVE DISPLAY IN SIMULATION OF INERTIA, which was filed on Jul.22, 2005 in the names of W. Daniel Hillis and Bran Ferren, and issued onMar. 15, 2011 as U.S. Pat. No. 7,907,124. The '186 application acontinuation-in-part of the following earlier-filed and co-pendingapplication and claims the benefit thereof in accordance with 35 USC120: U.S. patent application Ser. No. 10/913,105 entitled TOUCHDETECTING INTERACTIVE DISPLAY, which was filed on Aug. 6, 2004 in thenames of W. Daniel Hillis, Bran Ferren, and Ryan McKinley, and issued onJun. 1, 2010 as U.S. Pat. No. 7,728,821. All of the foregoingapplications are fully incorporated herein by this reference thereto.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to interactive touch and force-sensitivedisplays controlled through user gestures. More particularly, theinventions concerns the machine-implemented identification of particularhuman gestures from points of contact and applied force, and theimplementation of predetermined machine-implemented actionspre-associated with the gestures.

2. Description of the Related Art

There are many situations in which people wish to collaboratively andinteractively explore image-based data, for example by mutually lookingat and manipulating a paper map. With large, table-like touch sensitivedisplays, a group of users can jointly view imagery. This is preferableto huddling around a single workstation or sitting at separateworkstations and conversing by phone or email.

Imagery on table-like touch sensitive interactive displays is typicallymanipulated by a single user, either seated at a separate workstation orusing physical controls on the display. For example, the National Centerfor Supercomputing Applications developed an interactive display withprojected imagery, which is operated by a single user at a nearbycontrol console. Although this approach has some benefits, some teamsmight find that this scheme does not permit them to interact with thedisplay as intuitively and meaningfully as desired.

Another approach is Sony's SmartSkin interactive display, in whichoperators manipulate a computer's display using a limited set ofgestures such as panning and rotation. If utilized for certainapplications such as geographic imagery, however, users might find thatthe Sony system lacks image manipulating gestures that are sufficientlycomplex and intuitive. Consequently, users might experience a protractedlearning curve, for example, by having to remember complicated interfacecommands.

Consequently, known multi-user touch sensitive interactive displays arenot always completely adequate for all applications due to certainunsolved issues.

SUMMARY OF THE INVENTION

Broadly, one aspect of this disclosure concerns a method and apparatusfor operating a multi-user interactive display system including adisplay having a touch-sensitive display surface. A position is detectedof each contact site at which the display surface experiences externalphysical contact. Each contact site's position history is utilized tocompute velocity data for the respective contact site. At least one ofthe following is utilized to identify occurrence of one or more usergestures from a predetermined set of user gestures: the positionhistory, the velocity data. Each user gesture corresponds to at leastone predetermined action for updating imagery presented by the displayas a whole. Action is commenced corresponding to the identified gesture.Responsive to a user gesture terminating with a nonzero velocity acrossthe display surface, action is corresponding to the gesture is continuedso as to simulate inertia imparted by said gesture.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram of the hardware components andinterconnections of an interactive multi-user touch sensitiveinteractive display system.

FIG. 1B is a plan view showing several users operating an interactive,touch detecting display.

FIG. 1C shows a side view of an interactive, touch detecting, tabletopprojection display.

FIG. 1D is a block diagram of a digital data processing machine.

FIG. 1E shows an exemplary signal-bearing medium.

FIG. 1F shows exemplary logic circuitry.

FIG. 2 is a flowchart of a sequence for operating a multi-user touchsensitive interactive display system.

FIG. 3A shows a schematic and a representation of a panning gesture.

FIG. 3B shows a schematic and a representation of a zoom-in gesture.

FIG. 3C shows a schematic and a representation of a zoom-out gesture.

FIG. 3D shows a schematic and a representation of a rotation gesture.

FIG. 4 shows a gesture performed to rotate imagery about the center of acircular or semicircular display.

DETAILED DESCRIPTION

The nature, objectives, and advantages of the invention will become moreapparent to those skilled in the art after considering the followingdetailed description in connection with the accompanying drawings.

Hardware Components & Interconnections

Overall Structure

One aspect of the present disclosure concerns an interactive touchdetecting display system, which may be embodied by various hardwarecomponents and interconnections, with one example being described inFIG. 1A. The system 120 includes a table 122 with a display surface 124,computer 126, and projector 128. The projector 128 projects imagery uponthe display surface 124 under direction of the computer 126. As oneexample, the system 120 may be implemented by a touch detectinginteractive display as disclosed in U.S. patent application Ser. No.10/913,105, the entirety of which is incorporated by reference.

The table 122 detects touch input from human users as applied to thedisplay surface 124, and provides signals to the computer 126representing the position, size, timing, and other characteristics ofuser touch. Optionally, the table 122 may also detect applied force.Based upon this information, the computer 126 identifies one or moreuser gestures from a predefined set of defined gestures, and furtheridentifies an action associated with each identified gesture. In thisrespect, the computer 126 includes a gesture dictionary 126 a, listingof actions 126 b, and mapping 126 c between gestures and actions. Thecomputer 126 interprets the table 122's output by utilizing thedictionary 126 a to identify the gesture performed by the user. Thecomputer 126 then carries out appropriate action 126 c corresponding tothe user-performed gesture. The actions 126 c comprise, for example,predetermined machine executable operations for updating imagerypresented by the display.

The presently described embodiment of the system 120 facilitates usermanipulation of the projected imagery as a whole, for example, throughoperations such as panning, zooming, rotating, and the like. Thiscontrasts with personal computer applications, which utilize numerousseparately movable icons. Still, the system 120 may utilize one or moreperipheral menus or other control interfaces to support usermanipulation of the subject imagery. Accordingly, the system 120 isparticularly well suited to hands-on, intuitive, collaborative,multi-user study and manipulation of a large unitary item of imagerysuch as a photograph or map, presented upon the display 124.

In this respect, FIG. 1B shows several users operating an interactive,touch detecting display 11. The users 10 surround the display 11, suchthat each user can view the display surface 12, which shows imagery ofinterest to the users. For example, the display may present GeographicInformation System (GIS) imagery characterized by geographic 13,economic 14, political 15, and other features, organized into one ormore imagery layers. Because the users can comfortably surround and viewthe display, this readily facilitates group discussion and interactionwith the display. In the example of FIG. 1B, a user 16 has gestured 17by placing his fingertips on the display surface and moving them in anoutwardly separating manner. As discussed in greater detail below, thisparticular gesture 17 is associated with a zoom-in command. When thecomputer 126 performs a zoom-in command, it directs the system 120 toprovide a closer, more detailed view of the displayed imagery.

FIG. 1C shows a side view of the components 124, 128. The displaysurface is a horizontally oriented, planar projection surface 21supported by a table-like structure 22. The structure in this examplesupports the projection surface at waist level for adult users, allowingthe users to view and touch the entirety of the projection surfacecomfortably. The displayed imagery is generated by a projector 23located above and projecting 24 downward onto the projection surface.

While projection from above onto a horizontally oriented display isillustrated, this disclosure also contemplates other display surfaceorientations, projector configurations, and display technologies. Forexample, a horizontally oriented rear-projection surface may be used asthe display surface, with the projector mounted below the displaysurface and projecting upward. This approach eliminates shadows thatcould be generated when a user may position his body between theprojector and the projection surface. The display may also be mounted ina vertical orientation and affixed to a wall or other supportingstructure. A non-projection embodiment is also contemplated, employingthin profile display technologies such as LCD's, OLED's, or plasmadisplays. Despite the foregoing examples, skilled artisans in therelevant areas of technology will appreciate a further variety ofsuitable display technologies.

A possible consequence of the horizontal orientation of the displaysurface is a natural inclination of users to rest a hand on theprojection surface for support, especially when leaning forward to pointto objects near the center of the projection surface. To avoiderroneously interpreting such contact with the display as a gesture, theprojection surface may be surrounded by a small railing (not shown).This railing provides a visual cue that discourages users from leaningonto the display, and also provides structural support should a userlean in toward the center of the display.

Referring to FIG. 1A, the table 122 may employ various approaches todetect of when and where a user touches the display surface. In oneembodiment, a set of infrared emitters and receivers (not shown) isarrayed around the perimeter of the display surface 124, oriented suchthat each emitter emits light in a plane a short distance above thedisplay surface. The table 122 determines the location where the user istouching the projection surface by considering whether each emitters isoccluded (or not) as viewed from each of the receivers. As one example,a configuration incorporating a substantially continuous set of emittersaround the perimeter and three receivers, each positioned in a corner ofthe projection surface, may be utilized to resolve multiple locations ofcontact.

As an alternative, the table 122 may employ a resistive touch pad, suchas those commonly used in laptop computers, placed beneath the displaysurface 124, which is flexible. The resistive touch pad comprises twolayers of plastic that are separated by a compressible insulator such asair, and a voltage differential is maintained across the separatedlayers. When the upper layer is touched with sufficient pressure, it isdeflected until it contacts the lower layer, changing the resistivecharacteristics of the upper to lower layer current pathway. Byconsidering these changes in resistive characteristics, the computer 126can determine the location of contact.

In yet another embodiment, the table 122 employs a thin layer of liquidcrystal film or other material that changes optical properties inresponse to pressure. The thin layer is placed beneath the displaysurface 124, which is flexible. One or more video cameras trained on theunderside of the material capture the changes in optical properties thatoccur when a user touches the projection surface and therefore appliespressure to the thin layer. The location of contact is then determinedby using the computer 126 to analyze the video camera images.

In another embodiment, the table 122 employs ultrasound used to detectcontact information. Another embodiment uses capacitive touch pads, withone example being the Synaptics TouchPad™ product. A variety ofcapacitive touch pads are available commercially, and described invarious publications. Furthermore, the table 122 may employ acombination of some of the foregoing schemes, such as IR together withultrasound.

In any case, the detection scheme employed by the table 122 periodicallyprovides a machine readable output signal to the computer 126, which isrepresentative of the time and location of user contact with the displaysurface 124. In one embodiment, the table 122 signifies time byproviding a signal representative of the timing of user contact; inanother embodiment, the table 122 indicates timing by providing itslocation output signal in real-time. In turn, the computer 126 analyzesthe information from the table 122 to identify user gestures. Dependingupon the implementation, the table output may comprise a raw signalcorresponding to the physics of the detection mechanism, or a morerefined signal indicative of actual contact position. Thus, the computer126 may further serve to interpret the table's output to develop aCartesian or other representation of touch position.

As an optional enhancement, the display surface 124 may be mounted onload cells or other devices that sense force of the user contact on thedisplay surface 124. In this embodiment, the table 122 additionallyprovides the computer 126 with a signal representing the applied force.As described in greater detail below, the computer 126 may employ thedetected force to supplement the identification of gestures. Oneexample, illustrated below in greater detail, permits the user to applyforce to slow imagery that has been set in motion using simulatedinertia. Similarly, the computer 126 may also use force intensity todetermine the gain or attenuation applied to the velocity used to carryout the identified gestures.

Exemplary Digital Data Processing Apparatus

Data processing entities such as the computer 126 may be implemented invarious forms. One example is a digital data processing apparatus, asexemplified by the hardware components and interconnections of thedigital data processing apparatus 100 of FIG. 1D.

The apparatus 100 includes a processor 102, such as a microprocessor,personal computer, workstation, controller, microcontroller, statemachine, or other processing machine, coupled to a storage 104. In thepresent example, the storage 104 includes a fast-access storage 106, aswell as nonvolatile storage 108. The fast-access storage 106 maycomprise random access memory (“RAM”), and may be used to store theprogramming instructions executed by the processor 102. The nonvolatilestorage 108 may comprise, for example, battery backup RAM, EEPROM, flashPROM, one or more magnetic data storage disks such as a hard drive, atape drive, or any other suitable storage device. The apparatus 100 alsoincludes an input/output 110, such as a line, bus, cable,electromagnetic link, or other means for the processor 102 to exchangedata with other hardware external to the apparatus 100.

Despite the specific foregoing description, ordinarily skilled artisans(having the benefit of this disclosure) will recognize that theapparatus discussed above may be implemented in a machine of differentconstruction, without departing from the scope of the invention. As aspecific example, one of the components 106, 108 may be eliminated;furthermore, the storage 104, 106, and/or 108 may be provided on-boardthe processor 102, or even provided externally to the apparatus 100.

Logic Circuitry

In contrast to the digital data processing apparatus discussed above, adifferent embodiment of this disclosure uses logic circuitry instead ofcomputer-executed instructions to implement processing entities of thesystem 120. FIG. 1F shows exemplary logic circuitry 140. Depending uponthe particular requirements of the application in the areas of speed,expense, tooling costs, and the like, this logic may be implemented byconstructing an application-specific integrated circuit (ASIC) havingthousands of tiny integrated transistors. Such an ASIC may beimplemented with CMOS, TTL, VLSI, or another suitable construction.Other alternatives include a digital signal processing chip (DSP),discrete circuitry (such as resistors, capacitors, diodes, inductors,and transistors), field programmable gate array (FPGA), programmablelogic array (PLA), programmable logic device (PLD), and the like.

Operation

Having described the structural features of the present disclosure, theoperational aspect of the disclosure will now be described. As mentionedabove, one operational aspect of this disclosure generally involves theidentification of particular touch-based user gestures from points ofcontact, velocity, and/or applied force, and implementing ofpredetermined actions associated with the gestures. Although the presentinvention has broad applicability to touch based computing systems, theexplanation that follows will emphasize the application of FIGS. 1A-1Cin order to explain a useful and tangible example; no limitation isintended by use of this example.

Signal-Bearing Media

Wherever the functionality of any operational components of thedisclosure is implemented using one or more machine-executed programsequences, these sequences may be embodied in various forms ofsignal-bearing media. In the context of FIG. 1D, such a signal-bearingmedia may comprise, for example, the storage 104 or anothersignal-bearing media, such as a magnetic data storage diskette 130 (FIG.1E), directly or indirectly accessible by a processor 102. Whethercontained in the storage 106, diskette 130, or elsewhere, theinstructions may be stored on a variety of machine-readable data storagemedia. Some examples include direct access storage (e.g., a conventional“hard drive”, redundant array of inexpensive disks (“RAID”), or anotherdirect access storage device (“DASD”)), serial-access storage such asmagnetic or optical tape, electronic non-volatile memory (e.g., ROM,EPROM, flash PROM, or EEPROM), battery backup RAM, optical storage(e.g., CD-ROM, WORM, DVD, digital optical tape), or other suitablesignal-bearing media including analog or digital transmission media andanalog and communication links and wireless communications. In oneembodiment, the machine-readable instructions may comprise softwareobject code, compiled from a language such as assembly language, C, etc.

Logic Circuitry

In contrast to the signal-bearing medium discussed above, some or allfunctional components may be implemented using logic circuitry such as140 (FIG. 1F), instead of using a processor to execute instructions.Such logic circuitry is therefore configured to perform operations tocarry out the method of the disclosure. The logic circuitry may beimplemented using many different types of circuitry, as discussed above.

Overall Sequence of Operation

FIG. 2 shows a sequence 200 to illustrate one example of the methodaspect of this disclosure. Broadly, this sequence serves to detect andanalyze contact points, history, velocity, and/or applied force torecognize user application of predefined touch-based user gestures, andthereafter implement predetermined actions pre-associated with therecognized gestures. As described in further detail below, optionalfeatures such as inertia, touch initiated object slowing, friction, andothers may be implemented. For ease of explanation, but without anyintended limitation, the example of FIG. 2 is described in the contextof the interactive touch input system of FIGS. 1A-1C.

Broadly, the steps 202, 204, 206 run continuously to process usercontact with the display surface 124 as it occurs. Steps 202, 204, 205therefore serve to analyze contact occurring when the user contacts thesurface 124 at one or more contact regions utilizing one or morefingers, hands, arms, etc. As explained in greater detail below, step208 analyzes the history of position, velocity, force, and other touchcharacteristics to recognize when the user has performed a recognized“gesture.”

The sequence 200 is now described in greater detail. As an example, thesequence 200 may be initiated upon boot up, reconfiguration,initialization, or other startup of the system 120. In step 201, theuser initiates (and the display/computer detects) the user's physicalcontact with the display surface 124. Without any intended limitation,the illustrated embodiment of the sequence 200 performs one instance ofthe (repeating) steps 202-204 for each such contact initiated. Thecontact of step 201 is referred to as the “current” contact. In onegesture recognition scheme, the computer 126 tracks a predeterminednumber of distinct contact locations (such as two). If the computeridentifies another contact location (such as a third), the computer 126ignores it until the user releases a sufficient number of the existingcontact locations.

In step 202, the table 122 detects and monitors the position, size,shape, and timing of the current contact region. Namely, the table 122provides a machine readable output to the computer 126, which isrepresentative of the position, size, shape, and timing of each contactregion, or contains information from which this information can becalculated or derived. The timing output may be satisfied, for example,by the table 122 providing its output in real time. Also in step 202,the computer 126 stores a position history for each contact region. Theposition history provides a record of how each contact region moves orand/or changes shape over time.

In step 204, the computer 126 computes and monitors the velocity (ifany) of the subject contact that is occurring by analyzing the contact'sposition history. The computed velocity may comprise an instantaneousvelocity, average velocity over some or all of the past, moving average,or other suitable computation.

In step 206, the table 122 detects and monitors the force by which thecurrent user contact is being applied. As a specific example, this mayoccur by the table 122 detecting applied pressure of the current contact(utilizing a mechanism such as load cells, solid state force sensors, orother devices), or by assuming that applied force increases or decreasesproportionally to the size of the contact region. To provide someexamples, step 206 may be performed concurrently with step 202, inseries (as shown), or omitted entirely. Also in step 206, the table 122provides a machine-readable output to the computer 126, this signalrepresenting the detected force or containing information by which forcecan be derived or computed.

In step 208, the computer 126 determines whether activity of the currentcontact matches a predetermined pattern, and therefore constitutes a“gesture.” Step 208 repeats continually, utilizing some or all of theposition, position history (movement), velocity, and force informationfrom steps 202, 204, 206. More particularly, in step 208 the computer126 compares the history of contact position, size, movement, velocity,and/or force to the dictionary 126 a of predetermined gestures todetermine if the user has performed any of these gestures.

As long as the current contact continues, but no gesture has beendetected, step 208 repeats (via 208 a). If the current contact ends butno gesture is detected (208 b), then the computer 126 may optionallyprovide feedback to the user that an attempted gesture was notrecognized (step 209). Feedback may be provided, for example, by audiblealert, visual alert, error log, etc. In contrast, if step 208 detectsthat the user has initiated a gesture (208 c), the computer in step 214utilizes the mapping 126 c to identify the action 126 b associated withthe gesture that was identified in step 208. As mentioned above, thepredefined actions include various machine implemented operations forupdating the presentation of imagery by the display. In one embodiment,gestures are both identified (208) and associated (214) with displaycontrol commands via a single procedure.

After step 214, the computer 126 initiates performance of the identifiedaction (step 216). As described in greater detail below, some examplesof actions 126 b include panning, zooming, rotating, and the like. Thus,step 216 starts the requested pan, zoom, rotate, or other operation.

In step 218, the computer/display detects that the current gesture hasended because the user terminated contact with the display. In a simpleembodiment, the computer 126 may respond to termination of the currentgesture by ending the associated action (step 220). However, bysimulating physical properties, such as inertia and friction, the system120 can more closely approximate the look and feel of manipulating aphysical object. An important consequence of these properties is thatmotion of the displayed imagery can continue, and subsequently cease,after the initiating points of contact are removed. Therefore, in step218 the computer 126 considers whether the gesture terminated with anon-zero velocity. In other words, step 218 determines whether, at themoment the user ended the current gesture by terminating contact withthe display surface, the contact region was moving. Step 218 mayconclude that the gesture ended with motion if there was any motionwhatsoever, or step 218 may apply a predetermined threshold (e.g., oneinch per second), above which the contact region is considered to bemoving.

If the current gesture ended with a zero velocity (or a nonzero velocitythat did not meet the threshold), then step 218 progresses (via 218 a)to step 220, where the computer 126 terminates the action beingperformed for the subject gesture. In contrast, if the current gestureended with a nonzero velocity, step 218 advances (via 218 b) to step222, which executes the action in a manner that imparts inertia to theaction.

For example, if the action identified in step 214 was “rotate,” then thecomputer 126 in step 222 directs the projector 128 to additionallycontinue the requested rotation after the gesture terminates. In oneembodiment, the imparted inertia may be proportional to the nonzerovelocity at gesture termination (computed at 204), which may serve tosimulate continuation of the motion that was occurring when the gestureterminated.

Another example is where the computer 126 detects (FIG. 2, step 208)that the user has initiated a pan gesture by drawing a finger across thedisplay surface at a particular velocity, and lifted his/her finger fromthe surface while still moving (FIG. 2, step 218 b). With the optionalinertia feature enabled, the computer 126 continues (FIG. 2, step 222)to pan the imagery in the initiated direction at the velocity implied bythe gesture at the time the finger was lifted until a stopping orslowing naturally occurs (step 224). If the velocity when the finger waslifted is low, the computer 126 pans the display at a correspondinglyslow rate, useful for slowly panning across imagery. Alternatively, ifthe computer 126 detects a panning gesture terminated at a rapidvelocity, the computer 126 quickly translates the imagery in the desireddirection, without the need for repeated panning gestures to continuemovement. The computer 126 similarly recognizes user termination ofother gestures with residual velocity, such as rotation and zoom, withinertia continuing the appropriate motion until stopped.

With various techniques, the routine 200 may slow the imparted inertiaas illustrated by step 224. For example, without user contact, thecomputer 126 may slow the inertia at a predetermined rate to simulatefriction. As another example, upon new user contact after terminatingthe gesture with inertia, the computer 126 may (1) slow the inertia inproportion to force exerted by the user, the size of the contact area,or other properties, (2) abruptly terminate the inertia, thus bringingthe motion of the imagery to an immediate stop, (3) terminate theinertia and immediately impart a motion correlating with the newcontact, or (4) perform another action.

One example of a slowing gesture (step 224) comprises placing the fingeror hand on the display surface, as if stopping a spinning globe. Inresponse to this gesture, the computer 126 may slow movement at a ratethat is proportional to the force with which the gesture is applied orto the area of contact. For example, responsive to the user lightlytouching a finger, the computer 126 will cause “drag” and gradually slowthe motion. Likewise, responsive to a firmer touch or wider area ofcontact (such as a whole hand), the computer 126 more briskly slows themotion, or immediately stops entirely. This graduated response is usefulwhen, for example, the imagery is panning at high speed and the desiredlocation is approaching. Thus, the user can gently slow down the displaywith a light touch then press firmly when the location is reached. In analternative embodiment, the computer 126 ceases motion at the first tapor other touch.

In one embodiment, the computer 126 is responsive to user input toenable, disable, and/or adjust the above described inertia, friction,and such properties. For example, a simulated friction coefficientgoverns the degree to which the imagery motion slows over time. With thefriction coefficient is set to zero or inactive, the computer 126utilizes a simulated friction of zero, and continues motion at theinitiated velocity until stopped by the user through a stopping gesture.In contrast, with the friction coefficient set to a nonzero value, thecomputer 126 slows the motion of the imagery at the given rate. Thecomputer 126 may also recognize an adjustable threshold for determiningmotion (218) or no motion (218 b).

Some Exemplary Gestures

Pan

Panning is one exemplary gesture. If the computer 126 detects (step 214)that a user establishes and then moves a single contact location, thecomputer 126 executes (step 216) a panning movement, that is,translation of the imagery as a whole in the direction of the movement.FIG. 3A shows an example of this. The direction and rate of the pan isdetermined by the velocity of the contact site.

Zoom

Zoom is another exemplary gesture. If the computer 126 detects (step214) the user establishing two contact locations and initiating aseparating motion (FIG. 3B), the computer 126 executes (step 216) aninward zoom of the imagery as a whole. FIG. 3C shows a schematic and arepresentation of an outward zoom, comprising an approaching motion ofthe two contact locations.

Rotation

FIG. 3D shows a schematic and a representation of a rotation gesture,comprising a user establishing two contact locations and initiating arotary motion of the locations at a substantially fixed radius about acenter of rotation. When the computer 126 detects (step 214) a clockwise(or counterclockwise) motion, the computer 126 executes (step 216) acorresponding rotation of the imagery as a whole, about the center ofrotation.

Gesture Speed

As mentioned above, the computer 126 alters the display through zoom,pan, rotate, or other actions in step 216. In one embodiment, the rateat which such display changes occur may be proportional to the magnitudeof the velocity of the contact point within the gesture, as measured instep 204. For those gestures consisting of two contact points, thecomputer 126 may consider the velocity of either one of the contactpoints, an average velocity, etc. The computer 126 may further apply again or attenuation to the resulting magnitude to provide the desiredbalance of speed and precision in display control.

Optionally, the computer 126 may further alter the rate of displaychanges in proportion to the force with which a gesture is applied. Forexample, the computer 126 may carry out the action associated with agesture more quickly in response to a greater input force.

Combined Gestures

In addition to the basic motions described, the computer 126 mayrecognize combined gestures to effect more complicated changes in thedisplayed imagery. For example, the computer 126 may be programmed torecognize (214) the user's establishing a right and a left contactlocation and initiating an offset separating motion, upward on the rightand downward on the left. In response, the computer 126 performs (step216) a combined inward zoom and counterclockwise rotation.

To avoid user confusion or disorientation when combined gestures areattempted, such as an imperfect attempt to simply zoom inward, thecomputer 126 may be programmed as discussed above to provide feedback(step 209) to the user when contact terminates without defining arecognized gesture. As an alternative, the computer 126 may interpretthe combined gesture as one or the other of the attempted gestures,insofar as one can be identified.

Additional Gestures

Although the foregoing description provides an exemplary set of basicgestures, those skilled in the art will appreciate that many additionalgestures may be devised, and different commands may be associated withthe existing or additional gestures. For example, certain gestures maybe desirable based on the geometry of the touch display.

For example, the computer 126 may recognize (step 214) a “lazy Susan”gesture, to which the computer 126 responds (step 216) by rotating thedisplayed imagery as a whole upon a predetermined point of rotation. Inone example, the touch table may utilize a circular or semicircularconfiguration (either by computer manipulated display or by physicalshape). In this case, the computer 126 responds to any linear or arcinggesture to rotate the display about a predetermined point, as if settinga rotating disk or “lazy Susan” in motion. Optionally, the computer 126may limit recognition of the lazy Susan gesture to the gesturesperformed at the outer periphery of the touch table.

In another embodiment, the touch table is not pre-configured in acircular or semicircular configuration. In this environment, thecomputer 126 recognizes an arc-shaped gesture or other sweepingsemicircular motion 40 (FIG. 4) as the lazy Susan gesture. Optionally,the computer 126 may limit recognition of this gesture to thecircumference of the table. The resulting action rotates imagery 41about the display center, simulating a physical rotation of theunderlying display surface.

Other Embodiments

While the foregoing disclosure shows a number of illustrativeembodiments, it will be apparent to those skilled in the art thatvarious changes and modifications can be made herein without departingfrom the scope of the invention as defined by the appended claims.Furthermore, although elements of the invention may be described orclaimed in the singular, the plural is contemplated unless limitation tothe singular is explicitly stated. Additionally, ordinarily skilledartisans will recognize that operational sequences must be set forth insome specific order for the purpose of explanation and claiming, but thepresent invention contemplates various changes beyond such specificorder.

In addition, those of ordinary skill in the relevant art will understandthat information and signals may be represented using a variety ofdifferent technologies and techniques. For example, any data,instructions, commands, information, signals, bits, symbols, and chipsreferenced herein may be represented by voltages, currents,electromagnetic waves, magnetic fields or particles, optical fields orparticles, other items, or a combination of the foregoing.

Moreover, ordinarily skilled artisans will appreciate that anyillustrative logical blocks, modules, circuits, and process stepsdescribed herein may be implemented as electronic hardware, computersoftware, or combinations of both. To clearly illustrate thisinterchangeability of hardware and software, various illustrativecomponents, blocks, modules, circuits, and steps have been describedabove generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the present invention.

The various illustrative logical blocks, modules, and circuits describedin connection with the embodiments disclosed herein may be implementedor performed with a general purpose processor, a digital signalprocessor (DSP), an application specific integrated circuit (ASIC), afield programmable gate array (FPGA) or other programmable logic device,discrete gate or transistor logic, discrete hardware components, or anycombination thereof designed to perform the functions described herein.A general purpose processor may be a microprocessor, but in thealternative, the processor may be any conventional processor,controller, microcontroller, or state machine. A processor may also beimplemented as a combination of computing devices, e.g., a combinationof a DSP and a microprocessor, a plurality of microprocessors, one ormore microprocessors in conjunction with a DSP core, or any other suchconfiguration.

The steps of a method or algorithm described in connection with theembodiments disclosed herein may be embodied directly in hardware, in asoftware module executed by a processor, or in a combination of the two.A software module may reside in RAM memory, flash memory, ROM memory,EPROM memory, EEPROM memory, registers, hard disk, a removable disk, aCD-ROM, or any other form of storage medium known in the art. Anexemplary storage medium is coupled to the processor such the processorcan read information from, and write information to, the storage medium.In the alternative, the storage medium may be integral to the processor.The processor and the storage medium may reside in an ASIC.

The previous description of the disclosed embodiments is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these embodiments will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other embodiments without departing from thespirit or scope of the invention. Thus, the present invention is notintended to be limited to the embodiments shown herein but is to beaccorded the widest scope consistent with the principles and novelfeatures disclosed herein.

The invention claimed is:
 1. A computer implemented method performed ina system including a processor coupled to digital data storage and adisplay having a touch-sensitive display surface, the method comprisingthe tasks of: in the digital data storage, storing a record defining acollection of multiple user gestures, each user gesture executable bytouching the display, and further storing for each user gesture anassignment of one or more of multiple prescribed operations of modifyingsubject matter presented by the display; for each of one or more touchesexperienced by the display surface, the processor determining themagnitude of the touch upon the display surface; based on one or moreprescribed properties of the one or more touches experienced by thedisplay surface, the processor identifying from the collection of usergestures at least one user gesture executed by the one or more touches;the processor identifying the one or more prescribed operations assignedto the executed user gesture, and causing the display to modify thesubject matter presented by the display according to the identified oneor more operations; and where the tasks are further performed accordingto any or both of: (1) the identification of the executed user gestureis performed based on properties including the determined magnitude ofthe one or more touches; (2) as to the manner in which the subjectmatter presented by the display is modified according to the identifiedone or more operations, said manner is further responsive to thedetermined magnitude of the one or more touches.
 2. The method of claim1, wherein said magnitude comprises any of: a current length, a currentarea, a current intensity, a current force, a length history, an areahistory, an intensity history, and a force history.
 3. At least onenon-transitory computer-readable storage medium containing a program ofmachine-readable instructions executable by a digital data processingmachine to perform tasks for operating an interactive display systemincluding a processor coupled to digital data storage and a displayhaving a touch-sensitive display surface, where the digital data storagecontains a record defining a collection of one or more user gestures,each user gesture executable by touching the display, and where thedigital data storage further contains for each user gesture anassignment of one or more prescribed operations of modifying subjectmatter presented by the display, where the tasks comprise: for each ofone or more touches experienced by the display surface, the processordetermining the magnitude of the touch upon the display surface; basedon one or more prescribed properties of the one or more touchesexperienced by the display surface, the processor identifying from thecollection of user gestures at least one user gesture executed by theone or more touches; the processor identifying the one or moreprescribed operations assigned to the executed user gesture, and causingthe display to modify the subject matter presented by the displayaccording to the identified one or more operations; and where the tasksare further performed according to any or both of: (1) theidentification of the executed user gesture is performed based onproperties including the determined magnitude of the one or moretouches; (2) as to the manner in which the subject matter presented bythe display is modified according to the identified one or moreoperations, said manner is further responsive to the determinedmagnitude of the one or more touches.
 4. The method of claim 3, whereinsaid magnitude comprises any of: a current length, a current area, acurrent intensity, a current force, a length history, an area history,an intensity history, and a force history.